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•  Background  -  Evaluating  CMIST 

•  What  is  CMIST? 

•  Evaluation  goals 

•  Pacifica  overview 

•  PMESII  Modeling 

•  Modeling  methodology 

•  PsyOps/MCO  model 

•  Experimental  Results 

•  Running  different  COAs 

•  COA  Comparison  &  Drilldown 

•  Lessons  Learned 

•  Modeling  process 

•  Tool  insights 

•  Future  Work 
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CM  1ST  Overview 


•  Multi-Methodology  M&S  IDE 

•  Modeling  tools  for  authoring, 
debugging,  simulation, 
visualization,  exporting 

•  Developer  tools  for  authoring 
new  ontologies,  integrating 
native  simulation  tools 

•  Built  on  proven  open 
standards  and  C/GOTS 

•  Plug-ins  to  the  widely-used 
Eclipse  Java  IDE 

•  XML,  XSD,  UML,  XDA,  OAT 
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Families  of  Simulation  Methodologies 

•  Methodologies  currently  Incorporated  In  tool 

•  Cause-effect  (AFRL  CAT) 

•  System  Dynamics  (U.  Berkeley’s  Ptolemy  II) 

•  Agent-based  modeling  (TILab’s  JADE) 

•  Methodologies  being  incorporated  this  year 

•  Bayesian  Knowledge  Bases  (E.  Santos,  Dartmouth  College) 

•  Colored  Petri  Nets 

•  Network  links  to  data  sources  (not  really  a  methodology) 

•  Extensible  libraries 

•  Transforms  for  cross-model  data  exchange 

•  Interaction  patterns  for  synchronizing  models  with  disparate  time  management 
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CMIST  Evaluation  Goals 

•  Mission  Statement 

•  Evaluate  and  assess  the  utility,  robustness  and  reievance  of  CMIST  to 
provide  command-level  decision  makers  the  capability  to  anticipate  and 
shape  future  battlespaces 

•  Resources 

•  CMIST  1 .0  from  previous  effort  (see  MORSS  75th  EBO  SS) 

•  Unclassified  Pacifica  dataset  (use  and  augment  as  needed) 

•  Specified  Tasks 

•  Identify  scenario-specific  requirements,  data  and  models 

•  Define  models  and  input  data  needed  to  apply  CMIST 

•  Exercise  CMIST  PMESII  modeling  capabilities  in  a  realistic  context 

•  Evaluate  the  performance,  efficacy  and  utility  of  CMIST 

•  Strengths  and  weaknesses 

•  Gauge  appropriateness  for  COA  generation  and  assessment 

•  Endstate 

•  CMIST  demonstration  and  tutorial  using  the  Pacifica-based  scenario 

•  Final  report  on  evaluation  criteria  and  metrics,  planning  process,  tool  operation 
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PMESII  Modeling  Methodology 


1 .  Develop  an  issue  statement 

2.  Identify  key  variables  &  other  data 

3.  Define  reference  behavior  patterns 

4.  Develop  a  structural  dynamic  hypothesis 

5.  Build  a  simulation  model 

6.  Iterate,  improve,  and  implement 


Modeling  Focus:  PsyOps  impact  on  Major  Combat  Operations 
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Pacifica  Overview 


•  Red  strategy  (Califon) 

•  Goal:  Prevent  Nevidah  development  of 
port  of  Salt  Lake  City 

•  Actions:  Invade  Nevidah  to  take  mineral 
fields  with  conventional  military  forces 

•  Blue  strategy 

•  Phi:  Deterrence  through  defensive 
deployment,  Info  Ops,  COIN  Ops 

•  Ph  II  &  Ph  III  -  Major  Combat  Ops 

•  Ph  II:  Seize  the  Initiative  through  air 
superiority,  begin  offensive  counter-land 

•  Ph  III:  Decisive  Ops:  Attrit  Califon  army, 
WMD,  and  offensive  capability 

•  Ph  IV:  Transition:  Monitor,  support, 
redeploy 


BAE  SYSTEMS 
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1 .  Issue  Statement 

•  Models  will  compare  PsyOps  non-kinetic  strategies  with  a 
conventional  scenario  of  Major  Combat  Operations  (MCO) 

•  Focus  on  aggregate  values  such  as  Land  Area  and  number  of 
personnel,  but  create  individuals  such  as  Red  Commander  for 
decision-making 

•  Model  the  Califon  military  organization  featuring  three  levels  of 
hierarchy: 

•  Commander 

•  Staff 

•  Personnel. 

•  The  lower  two  levels  can  be  influenced  by  PsyOps  campaign, 
which  reduces  the  will  to  fight  of  Califon  personnel  and  staff 
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•  A  few  key  variables  are  listed  below.  During  this  stage  of  the 
modeling  process,  we  are  trying  to  outline  some  important 
features,  but  we  will  revisit  and  add  variables  as  we  go 

•  Califon  combat  power  (personnel  +  equipment) 

=  Califon  force  factor 

•  Blue  combat  power  (similar) 

•  Land  gained  by  Califon  (in  contested  PMF  region) 

•  Califon  personnel  with  will  to  fight 

•  Blue  personnel  and  equipment  losses 

•  Califon  decisions  (invade  or  not,  retreat,  etc.) 

•  Califon  Commander 
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Scenario  1  -  MCO  with  Close  Air  Support 


•  Califon  invades  Nevidah  on  day  2 

•  Califon  decisions  in  Cause-Effect  methodology  feeds  into  Agent  model  of 
Califon  Commander,  and  decision  output  of  military  action  and/or  surrender 
feeds  into  the  larger  System  Dynamics  model 

•  Nevidah  land  (mineral  fields)  under  Califon  control  is  a  major  indicator 

•  Blue  starts  MCO  on  day  30 

•  Blue  commanders  are  Agents  with  deterministic  decisions  which  are  played  out 
in  the  System  Dynamics  model 


June-2008  /  CMIST  /  Unclassified 


12 


UNCLASSIFIED 


BAE  SYSTEMS 


Scenario  2  -  MCO  with  PsyOps  only  targeting  Personnel 


•  Same  basic  structure  as  Scenario  1 

•  PsyOps  campaign  initiated  by  Joint  Commander  on  day  18 

•  Califon  Personnel  with  Will  To  Fight  targeted  by  PsyOps  campaign  (Holdouts 
are  not  targeted  by  campaign) 
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Scenario  3  -  MCO  &  PsyOps  targeting  Personnel  &  Staff 


•  Same  basic  structure  as  Scenario  2 

•  PsyOps  campaign  initiated  by  Joint  Commander  on  day  18 

•  Califon  Personnel  with  Will  To  Fight  and  Staff  with  Will  To  Fight  are  both 
targeted  by  PsyOps  campaign 

•  These  two  organizational  levels  interact  with  each  other  in  a  positive  feedback 
fashion,  resulting  in  much  faster  rates  of  demotivation 
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Callfon  Decisions  -  Cause  Effect  &  Agent  Model 


Models  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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Callfon  Force  Factor  &  Land  Gain  -  System  Dynamics  (SD) 


♦  SAM_destroyed_per_flight 


♦  adding_equipment  .  ^  ^ 

Califon_Equipment 

•  duuii  nj  ^ 

♦  destroying_SAMs  <  •  ^^gradmg  | 


♦  equipJoss_rate 


losing_equipment 


Blue_force_factor 


♦  total_equipment 


♦  Califon_ff_parameter 


total_Califon_personnel 


Califon_force_factor 

^  ratio_force_factors 

♦  effect_onjan<jjoss 


♦  effect_of_engagement_wiBh_blue_force_onJand_gain 
►  ff_ratio_and_blue_engagment_factor 

♦  Califon_gainingJand 


♦  Caljfon_desired_adjustment_bme  ♦  Califon_attempted_land_gain_rate 


-V  gaining 


♦  military_acbonJand_gajn 


♦  Califon_effort_towards_land_gain 


♦  area  of  Nevidah  mineral  fields 


♦  gapJnJand_controlled 


♦  bluejand_gain_rate 


♦  losingjand 


Models  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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Major  Combat  Operations,  Blue  Forces  -  SD  &  Agent 


♦  SAM_destroyed_per_flight  ♦  flight_per_aircraft_per_day 


adding  adding_air_power 


Air_Conimander 


>  destroying_SAMs 


♦  blue_air_flights 


♦  conduct_air_striLes  ^  3^,  ^^,,3  transform 


♦  SAMJaunches  ❖  aircraft_destroyed_per_SAM 


destroying_CAS 


❖  RedJnfluenced_by_PSYOPS 

^  psyops  transform 

>  adding_CAS 


♦  Blue_ff_para... 


Blue_force_factor 


♦  Blue_Land_Component 


♦  normal_rate_bl... 


■V-  Blue_Personnel_Deatlis 

<  deaths  ^  losing_blue_personnel  _ . 

V  Blue_Personnel 

ratio_force_factors 


❖  effect_on_blueJosses 


losing_blue_equip 


♦  normal_rate_bl... 


degrading 


Blue_Eqiilpment 


<>■  Joint_Conimander 


4^  Link 

4  PsyO|Jl_decision 


4  Link 

4  strike  DP  decision 


►  combat_command 


4  start  combat 


4  Ground.Commander 


4  Link 


4  combat  decision 
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PsyOps/MCO  Reference  Behavior  Patterns  -  1 

•  Reference  behavior  patterns  display  historical  or  expected  future 
values  for  a  variable  as  a  time-series 


•  Time  horizon  is  zero  to  60  days  into  the  future 

•  Red  line  shows  expected  behavior  pattern  for  Califon  if  no  Blue  intervention 

•  Blue  line  shows  expected  behavior  when  Blue  launches  Major  Combat  Operations 
-  increasingly  steep  losses  for  Califon 
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•  Force-on-Force  interactions  apply  attrition  rates  proportional  to  the 
force  factor  ratio  (Califon)  or  its  reciprocal  (Blue) 

•  Potential  for  spiral  downwards  when  your  force  factor  gets  smaller 


Blue  force  factor 

force  factor  ratio  =  - 

Califon  force  factor 


Models  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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PsyOps/MCO  Reference  Behavior  Patterns  -  2 


5000 


Red  &  Blue 
Personnel  Deaths 
during  Major 
Combat  Ops 

no  PsyOps 
campaign 


0 
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•  PsyOps  campaign  starts  demotivation  in  Califon  personnel  &  staff 

•  Which  then  self-reinforces  through  social  processes  such  as  word- 
of-mouth  and  observations  of  others  demotivating 


demotivated  by  Blue  PsyOps  campaign 


Models  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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Will  To  Fight  Demotivation  Structure  -  SD 


Positive  feedback  growth  loop  of  Demotivated  Personnel 


❖  fraction_of_total 


♦  demotivated_personnel*meeting_potential  ❖  normal_rate_of_meeting_people 


Models  presented  here  are  based  on  notional  or  open  source  information  june-2008  /  cmist  /  unclassified 
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Demotivation  Across  Caiifon  Organization  -  SD 


Models  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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Callfon  Will  To  Fight  /  Blue  MCO  Model 


Models  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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When  Califon  attacks  &  Probability  of  continuing  to  fight 


Data  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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Red  and  Blue  force  factors 


Data  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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Land  under  Califon  control 


Data  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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Total  Califon  personnel  (including  hoidouts) 


Data  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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Personnel  with  will  to  fight  (not  inciuding  hoidouts) 


—  Personnel_with_Will_To_Fight:  currentValue 


—  Personnel_with_Will_To_Fight:  currentValue 


COA  1  -  MCO  with 
Close  Air  Support 


—  Personnel wilh Will To Fight:  currentValue 


COA  2  -  MCO  + 
PsyOps  on  Personnel 


COA  3  -  MCO  +  PsyOps 
on  Personnel  &  Staff 


Data  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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Data  presented  here  are  based  on  notional  or  open  source  information  June-2008  /  cmist  /  unclassified 
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Modeling  Lessons  Learned 

•  Promising  initial  results  mixing  modeling  methodologies 

•  Successfully  demonstrates  CE-SD-Agent  interactions 

•  Combines  native  tools  with  discrete  vs.  aggregate  time  management 

•  Useful  results  on  Califon/Blue  MCO/PsyOps  model 

•  Demonstrates  ability  to  model  soft  factors,  such  as  psychological  influence 
campaign  on  adversary  and  the  resulting  social  dynamics 

•  Can  mix  probabilistic  Cause-Effect  trees  with  Agent  logic  structures  to  represent 
decision-making  under  uncertainty 

•  Can  test  assumptions,  both  structural  and  numerical,  in  experiments: 

•  What  happens  if  we  increase  the  effort  in  PsyOps?  (numerical) 

•  What  if  we  start  PsyOps  earlier,  or  later?  (numerical) 

•  What  outcomes  result  if  we  target  adversary  officers/staff  in  PsyOps  campaign  as  well 
as  their  personnel?  (structural) 

•  other  modeling  work 

•  Earlier  work  with  CMIST  generated  a  PMESII  notional  insurgency  model, 
demonstrating  effects  of  unemployment,  insurgent  recruitment,  and  impacts  of 
aid  programs 

•  Usable  for  modeling  DIME,  PMESII,  counter-insurgency,  information  ops,  etc. 
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Tool  Lessons  Learned 


•  CMIST  provides  a  rich  platform  for  PMESII  model  integration 

•  M&S  developers  IDE  to  rapidly  integrate  new  native  simulation  tools  (on 
the  order  of  a  few  weeks) 

•  Commander’s  IDE  for  rapid  model  authoring,  execution,  and  refinement 
(develop  simulating  models  in  as  little  time  as  a  few  hours) 

•  CMIST  lacks  some  features  of  more-developed  simulation  tools 

•  Syntax  checking  of  model  equations,  arbitrary  lookup  functions,  saving 
simulation  data  for  reuse  &  comparison,  multi-level  display  of  node-link 
models,  senstitivity  testing,  optimization,  etc. 

•  We  are  addressing  some  of  these  features  in  CyberCAFE 

•  Leveraging  open-source  community’s  work 

•  But  need  to  be  careful  to  mark  your  code  so  you  can  update  to  new 
releases 

•  Enthusiastic  response  from  Air  Force  SME;  potential  future  as 
modeling  tool  for  rapid  construction  of  low-overhead  models 
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Future  Work 


External  data  sources 


bomb  attacks 


CyberCAFE 


PMESII  Model 


Data  Node 
Public  Opinion 

— 

-  - 

^^Data  Node 

/- 


Commander  Agent  Model 


•Mental  Model 


Adversary  Intent  Model 


Mental  Model  Beliefs 


.  I  Observations  | 


Decisions 


Adversary  Communications 


^  Observations  | 
m  •  Objectives 


Blue  cyber  attack  on  red 
communication  infrastructure 


CyberCAFE  -  Cyber-PMESII  Commander’s  Assessment  of  Forecast  Effects 

•  Unified  cyber-PMESII  modeiing  environment  buiit  on  CMIST  IDE 

•  Intent  Modeling  for  friendly,  neutral,  adversary  agents 

•  Temporal  Bayesian  Knowledge  Bases  capture  beliefs,  rationales,  actions,  effects  (what,  why,  how) 

•  Embedded  agent  simulations  distinct  from  main  world  enable  complex  deception  strategies 

•  Colored  Petri  Nets  enable  information  flow  modeling  for  cyber-COA  analysis 

•  Connection  to  Live  Data  Sources,  Web-based  or  database  sources  to  inform  model  inputs 

•  Comparative  observational  data  helps  validate  model  outputs 
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